A Model-driven Performance Analysis and Deployment Planning for Real-time Stream Processing

Real-time stream processing in the cloud is gaining significant attention for its ability to mine massive amounts of data for a variety of applications, such as in reconnaissance missions or search-and-rescue operations. In cloud-based real-time streaming applications, dynamic resource management mechanisms are needed to support the real-time requirements of these applications. However, for any dynamic resource management technique to work, there is first a need to understand the controllable properties (or parameters) of the stream processing applications. Pinpointing these properties and separating them from the application-specific properties that cannot be controlled is hard and requires a scientific approach to obtain these insights. This paper presents a model-driven performance analysis approach for real-time streaming applications to pinpoint their controllable properties. The same modeling framework then makes deployment planning decisions, which is one dimension of dynamic resource management. The presented research is part of our larger effort towards a holistic framework to support real-time and dependable cloud-based applications.